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ABSTRACT 


A  large  number  of  ship  masters  gained 
their  first  exposure  to  the  problems  of 
transiting  Valdez  Narrows  during  re¬ 
search  performed  at  the  Maritime 
Administration's  Computer  Aided  Oper¬ 
ations  Research  Facility  (CAORF)  at 
Kings  Point,  New  York.  It  is  now 
appropriate  to  examine  the  effective¬ 
ness  of  this  simulator  experience. 
Whether  simulator  experience  and  real- 
world  experience  are  comparable  is  an 
issue  of  particular  relevance  to  those 
masters  who  have  participated  in  a 
familiarization  and  certification  pro¬ 
gram  dealing  with  pilotage  through  the 
Valdez  Narrows  to  the  Port  of  Valdez. 
The  present  study  was  designed  to  exa¬ 
mine  the  degree  to  which  simulator 
experience  received  at  CAORF  was 
effective  in  improving  pilotage  per¬ 
formance  compared  to  experience  re¬ 
ceived  in  real-world  situations.  Four 
groups  of  masters  were  tested  on  a 
simulated  pilotage  through  the  Valdez 
Narrows.  These  groups  consisted  of: 

o  Group  1  -  Masters  with  simulated 
Valdez  experience  and  real-world 
Valdez  experience.  (N=5) 


o  Group  2  -  Masters  with  no  simu¬ 
lated  Valdez  experience  but  with 
real-world  experience  in  the 
Valdez  pilotage  area.  (N=5) 

o  Group  3  -  Masters  with  simulated 
Valdez  experience  but  no  real- 
world  experience  in  the  Valdez 
pilotage  area.  (N=5) 

o  Group  4  -  Masters  with  no  simu¬ 
lated  Valdez  experience  and  no 
real-world  experience  in  the 
Valdez  pilotage  area.  (N=5) 

Performance  data,  measured  as  track 
deviations  and  percentage  of  time  out 
of  a  prescribed  tolerance  band,  indi¬ 
cated  equivalent  performance  for 
Groups  1,  2,  and  3,  and  significantly 
poorer  performance  for  that  group 
which  had  neither  simulator  nor  real- 
world  experience  in  the  Port  of  Valdez. 
The  results  indicated  that  experience 
with  the  CAORF  Valdez  operational 
exercises  was  as  effective  in  improving 
performance  as  was  real-world  experi¬ 
ence  in  the  Valdez  pilotage  area. 
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CHAPTER  1 


INTRODUCTION 


That  simulator  experience  can  provide 
valuable  assistance  in  the  acquisition  of 
marine-oriented  skills  has  traditionally 
been  accepted  in  the  maritime  industry. 
It  is  for  this  reason  that  the  Port  and 
Tanker  Safety  Act  of  1978  requires  the 
use  of  simulators  in  determining  stand¬ 
ards  relating  to  the  qualification  and 
training  of  crews. 

The  study  described  in  this  report  is  one 
of  a  series  of  training  and  certification 
criterion  programs  being  conducted  at 
the  Maritime  Administration's  Com¬ 
puter  Aided  Operations  Research  Facil¬ 
ity  (CAORF)  at  Kings  Point,  New  York. 
The  goal  of  this  program  is  to  develop 
performance  criteria  as  a  function  of 
simulator  experience  and  type  of  in¬ 
structional  process.  Eventually,  this 
and  other  studies  will  fulfill  an  execu¬ 
tive  directive,  issued  in  March,  1977, 
which  calls  for  the  upgrading  of  deck 
officer  certification  through  the  use  of 
simulators  and  training  programs. 

While  attempts  have  been  made  to  de¬ 
lineate  which  aspects  of  simulator 
experience  contribute  the  most  to  per¬ 
formance,  little  attention  has  been  paid 
to  an  empirical  determination  of  the 
value  of  simulator  experience  used  in 
conjunction  with  real-world  exposure. 
This  study  attempts  to  determine  the 
value  of  simulator  experience  to 
masters  who  either  have  or  have  not 
had  specifically  relevant  real-world  ex¬ 
perience.  In  addition,  the  study  also 
investigates  the  effectiveness  of  simu¬ 
lator  exposure  for  the  acquisition  of 
specific  shiphandling  skills  and  the  re¬ 
tention  of  such  skills 

It  is  essential  that  the  ppai  ability  of 
simulator  experience  and  real-world  ex¬ 


posure  be  ascertained,  if  simulator  ex¬ 
perience  is  to  be  incorporated  into 
training  and  certification  criterion  pro- 
grains  where  specific  functional  objec¬ 
tives  must  be  met.  It  is  also  necessary 
to  determine  the  most  effective  com¬ 
bination  of  simulator  experience  and 
on-the-job  training  in  the  acquisition  of 
shiphandling  skills.  In  the  present 
experiment  the  effects  of  previous  on- 
the-job  experience  and  simulator  ex¬ 
perience  are  separated  in  an  attempt  to 
compare  the  effect  of  simulator  experi¬ 
ence  to  that  of  real-world  exposure  on 
subsequent  shiphandling  performance. 

The  particular  shiphandling  perform¬ 
ance  objective  chosen  for  this  experi¬ 
ment  is  the  demonstration  of  the  know¬ 
ledge  and  skills  necessary  for  transiting 
the  Valdez  Narrows  in  Alaska.  The 
approach  to  and  departure  from  the 
Port  of  Valdez,  Alaska,  via  the  Valdez 
Narrows  is  indeed  an  important  and 
timely  concern  and  has  served  as  the 
basis  for  a  number  of  past  studies  con¬ 
ducted  at  CAORF.  These  studies  have 
already  yielded  results  immediately 
relevant  to  the  safe  transiting  of  ships 
in  the  Valdez  Narrows.  The  current 
study  continues  this  programmatic 
research. 

1.1  HISTORICAL  BACKGROUND 

Before  the  first  oil  through  the  Trans- 
Alaskan  Pipeline  reached  Valdez,  there 
was  great  concern  among  environment¬ 
alists,  state  officials,  and  the  Coast 
Guard  for  the  ecology  of  the  area  and 
the  safety  of  ships.  The  arrival  of 
VLCCs  and  other  ships  of  unprece¬ 
dented  size  in  these  waters  created 
concern  regarding  potential  groundings 
and  cargo  spillage.  Spillage  from  just 
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one  large  ship  could  ruin  the  Valdez 
Harbor,  which  is  part  of  one  of  the 
world's  great  salmon  fishing  areas. 

Valdez  has  several  natural  hazards  that 
must  be  taken  into  consideration  when 
attempting  to  assure  safe  passage 
through  the  Narrows  to  the  Port  of 
Valdez.  Of  greatest  concern  are  winds 
up  to  47  knots  which  have  been  re¬ 
corded  for  sustained  periods,  with  gusts 
of  75  knots  or  higher  at  times. 
"Williwaws,"  familiar  to  these  Arctic 
regions,  can  also  occur  suddenly  and, 
though  lasting  for  short  periods  of  time, 
can  reach  an  intensity  of  100  knots  or 
more.  Limited  visibility  occurs  during 
both  summer  and  winter  months  due  to 
precipitation  and  fog.  Middle  Rock, 
midway  across  the  Narrows,  would  be 
barely  visible  were  it  not  for  a  lighted 
30-foot  (9.14  m)  concrete  structure  in¬ 
stalled  by  the  Coast  Guard.  Beneath  it, 
rock  stretches  out  to  a  400-yard  base 
(366  m). 

Certain  precautionary  measures  have 
been  suggested  to  minimize  the  possi¬ 
bility  of  accidents  including  limiting 
traffic  to  one  way  only,  restricting  pas¬ 
sage  when  wind  speed  exceeds  40  knots, 
the  use  of  tug  escort  under  all  condi¬ 
tions,  and  the  recommendation  that  as 
part  of  the  licensing  requirement  pilots 
may  have  simulator  experience  at  an 
approved  facility.  With  this  in  mind, 
eight  Southwest  Alaska  pilots  partici¬ 
pated  in  an  experiment  carried  out  at 
CAORF  (Mcllroy,  W.,  "A  Review  of 
Valdez  Experiment,"  First  CAORF 
Symposium,  1977)  to  determine  condi¬ 
tions  for  safe  passage  through  the 
Narrows.  Other  objectives  of  that  ex¬ 
periment  were  to  determine  the  effec¬ 
tiveness  of  newly  developed  precise 
navigational  systems  and  the  develop¬ 
ment  of  pilot  responses  to  various 
power  and  steering  failures. 

The  visual  scene  presented  to  these 
pilots,  represented  a  realistic,  coloreu 


panoramic  view  of  the  entire  Valdez 
gaming  area  and  included  accurate 
placement  of  existing  navigational  aids 
and  other  prominent  features  used  as 
cues  by  pilots  negotiating  the  passage. 
In  addition,  both  wind  and  current  con¬ 
ditions  were  incorporated  into  this  real¬ 
time  simulation. 

The  performance  data  included  mea¬ 
sures  such  as: 

o  Track  deviations  from  an  ideal¬ 

ized  reference  track. 

o  Time  in  and  time  out  of  tolerance 

(+2.5  beam  widths). 

o  Ruuder  angle. 

o  Frequency  of  rudder  reversals. 

The  results  indicated  a  number  of  pro¬ 
blem  areas  for  pilots  such  as  response 
to  a  combined  power  and  steering  loss 
without  recovery. 

Additional  details  of  the  results  are 
beyond  the  scope  of  this  report,  but  this 
early  research  served  to  establish  the 
effectiveness  of  simulators  as  research 
and  as  training  instruments. 

As  a  result  of  the  above  experiment, 
the  United  States  Coast  Guard  intro¬ 
duced  a  program  which  allowed  40 
percent  of  a  pilot's  certification  re¬ 
quirements  to  be  obtained  by  way  of 
simulation  exercises.  This  led  to  the 
development  of  the  Valdez  Narrows 
operational  exercises  at  CAORF,  since 
this  was  the  only  U.S.  facility  to  have 
a  comprehensive  Valdez  Narrows  and 
harbor  data  base.  These  exercises, 
developed  during  1977,  investigated  the 
effectiveness  of  simulator  exercises  by 
providing  structured  experience  for 
masters  in  the  handling  of  a  165,000 
DWT  tanker  in  the  Valdez  Narrows 
under  a  variety  of  conditions. 


2 


The  subjects  for  these  exercises  were 
masters  from  various  shipping  com¬ 
panies  including  Sun  Oil  Company,  Cove 
Shipping,  Keystone  Shipping  Company, 
Marine  Transport  Lines,  the  Trinidad 
Corporation,  Gulf  Oil  Corporation,  and 
Exxon. 

For  these  operational  exercises,  the 
Valdez  Narrows  was  simulated  to  a  high 
degree  of  fidelity,  presenting  the  level 
of  information  necessary  for  masters  to 
successfully  conduct  the  operational 
exercises  along  the  USCG  recom¬ 
mended  track  for  ownship.  Besides  the 
visual  representation,  a  full-size  opera¬ 
tional  b.  idge  provided  appropriate  ship¬ 
handling  hydrodynamics  including  radar. 
In  addition,  various  environmental 
factors  were  incorporated  into  the  pre¬ 
sentations. 

The  operational  exercise  program  con¬ 
sisted  of  both  classroom  presentation  of 
related  topics  such  as  shiphandling 
when  failures  occur  and  hands-on  simu¬ 
lator  practice  of  the  concepts  taught  in 
the  classroom.  A  team  approach  was 
used:  while  the  bridge  team  worked  the 
problem  on  the  bridge,  an  observer 
team  monitored  its  performance  from  a 
remote  station  equipped  with  extensive 
closed-circuit  television  and  audio  cap¬ 
abilities.  The  program  for  each  subject 
was  conducted  over  a  five-day  period. 

Performance  data  were  displayed  dur¬ 
ing  each  run  for  the  benefit  of  the 
proctor  and  observer  teams  seated  at 
the  monitoring  station.  During  the 
program,  vessel  and  navigational 
parameters  indicative  of  shiphandling 
performance  were  automatically 
measured  and  recorded  for  analysis. 
The  operational  exercises  proved  to  be 
effective  as  shown  by  improvements 
during  the  five-day  period  in  track¬ 
keeping,  turning,  wind  compensation, 
radar  utilization,  and  the  handling  of 
propulsion  and  steering  failures. 


For  the  present  experiment,  some  of 
the  subjects  were  chosen  trom  masters 
who  had  partaken  in  the  Valdez 
Narrows  operational  exercises.  The 
subsequent  performance  of  these 
masters  conning  a  VLCC  through  the 
Valdez  Narrows  illustrates  the  degree 
to  which  previous  exposure  to  pilotage 
on  a  simulator  is  effective  in  improving 
performance  and  how  performance  is 
affected  by  experience  in  real-world 
situations.  The  requirements  of  the 
experiment  dictated  the  use  of  four 
groups  of  subjects  listed  below  and 
shown  in  Table  1. 

o  Group  1  -  Those  masters  who  had 
participated  in  the  Valdez  opera¬ 
tional  exercises  and  who  have  also 
had  real-world  experience  in  the 
Valdez  piloting  area. 

o  Group  2  -  Those  who  had  no 

Valdez  operational  exercise  ex¬ 
perience  but  have  had  real-world 
experience  in  the  Valdez  piloting 
area. 

o  Group  3  -  Those  who  had  Valdez 
operational  exercise  experience 
but  no  real-world  experience  in 
the  Valdez  piloting  area. 

o  Group  4  -  Those  who  had  no 

Valdez  operational  exercise  ex¬ 
perience  and  no  real-world  ex¬ 
perience  in  the  Valdez  piloting 
area. 

Comparison  between  Groups  1  and  2,  in 
which  real-world  experience  is  held 
constant,  permits  us  to  assess  the  train¬ 
ing  effectiveness  of  a  simulator. 

Comparison  between  Groups  2  and  3,  in 
which  real-world  Valdez  experience  is 
replaced  by  simulator  experience,  indi¬ 
cates  whether  the  degree  of  skill  ob¬ 
tained  with  a  simulator  is  greater  than, 
equal  to,  of  less  than  the  skill  derived 


TABLE  1.  EXPERIMENT  DESIGN 


Group 

Experience  with  Valdez  Narrows 

Real-World 

No  Real-World  Simulator  No  Simulator 

1  (N  =  5) 

o 

o 

2  (N  =  5) 

o 

o 

z 

II 

o  o 

4  (N  =  5) 

o  o 

from  observational  experience  transit-  stant,  and  comparisons  between  Groups 

ing  the  Narrows  in  the  real  world.  2  and  4,  in  which  the  absence  of  simula¬ 

tor  experience  is  held  constant,  show 
Comparison  between  Groups  3  and  4,  in  the  effects  of  observational  experience 

which  the  absence  of  real-world  Valdez  gained  by  transiting  the  Valdez 

experience  is  held  constant,  shows  the  Narrows, 
value  of  simulator  experience  alone  in 

improving  track-keeping  performance  The  results  of  all  comparisons  should 
under  simulated  conditions.  have  direct  bearing  on  certification  re¬ 

quirements,  retraining,  and  especially 
Comparison  between  Groups  1  and  3,  in  the  value  of  simulator  training  on  real- 
which  simulator  experience  is  held  con-  world  performance. 
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CHAPTER  2 


METHOD 


2.1  SOURCE  AND  BACKGROUND  OF 
SUBJECTS 

From  May,  1977  through  March,  1979, 
twenty  groups  of  masters  have  partici¬ 
pated  in  the  Valdez  Narrows  opera¬ 
tional  exercises.  The  program  of  oper¬ 
ational  exercises  familiarized  masters 
with  handling  ships  of  30,000  DWT  to 
250,000  DWT  through  the  Valdez 
Narrows.  The  primary  objectives  of  the 
program  were  to  develop  a  high  level  of 
expertise  in: 

o  Familiarization  with  the 

Narrows  and  associated  nav¬ 
igational  aids. 

o  Navigation  of  the  Narrows, 
following  a  USCG-recom- 
mended  track,  using  fixed 
navigational  aids  and  other 
visual  and  radar  cues. 

o  Shiphandling  under  various 

environmental  conditions. 

o  Shiphandling  under  various 

failure  conditions. 

A  total  of  90  individual  masters  have 
been  through  these  operational  exer¬ 
cises,  and  many  of  them  have  had  vary¬ 
ing  degrees  of  experience  with  the 
Valdez  harbor  itself.  In  view  of  this,  it 
was  found  that  there  were  numerous 
masters  who  fit  the  requirements  for 
participation  in  the  present  validation 
experiment. 

2.1.1  Test  Subject  Selection 

Test  subjects  were  selected  from  the 
aforementioned  pool  to  conform  with 
the  following  levels  of  experience. 


I.  Masters  with  previous  experience 
at  CAORF  in  the  Valdez  opera¬ 
tional  exercises  who  have  also  had 
extensive  experience  transiting 
Prince  William  Sound  and  the 
Valdez  Narrows  i.e.,  real-world 
experience.  (5  required) 

II.  Masters  with  previous  experience 
at  CAORF  in  the  Valdez  opera¬ 
tional  exercises  who  had  not  had 
any  real-world  experience  in 
Valdez  Narrows.  (5  required) 

III.  Masters  with  no  CAORF  experi¬ 
ence  in  the  Valdez  operational 
exercises  but  who  have  had  exten¬ 
sive  experience  transiting  Prince 
William  Sound  and  the  Valdez 
Narrows.  (5  required) 

IV.  Masters  with  backgrounds  similar 
to  the  rest  of  the  groups  (e.g. 
mean  years  at  sea,  types  of  li¬ 
censes  held)  but  with  no  CAORF 
Valdez  operational  exercise  ex¬ 
perience  and  no  real-world  exper¬ 
ience  in  the  Valdez  Narrows.  (5 
required) 


2.1.2  Familiarization 

All  test  subjects  were  provided  with 
preliminary  familiarization  before  par¬ 
ticipating  in  the  experiment.  Test 
subjects  were  shown  the  simulator 
wheelhouse,  equipment,  charts,  visual 
scene,  and  shiphandling  characteristics. 
They  were  also  given  the  opportunity  to 
observe  a  transit  on  the  bridge  of  the 
simulated  165,000  DWT  tanker  that 
would  be  used  in  the  experiment.  This 
allowed  each  subject  to  be  familiarized 
with  the  Valdez  gaming  area  and  the 
designated  track  which  they  would  be 
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required  to  follow.  The  track  was  illus¬ 
trated  on  the  appropriate  chart. 


2.2  ENVIRONMENTAL  SIMULATION 

2.2.1  Environmental  Data  Base  and 
Gaming  Area 

The  experiment  utilized  the  Computer 
Aided  Operations  Research  Facility 
(CAORF)  to  perform  the  tests.  A  des¬ 
cription  of  CAORF  is  given  in  Appendix 
A.  The  data  base,  which  already  exists 
at  CAORF,  consisted  of  the  area  sur¬ 
rounding  the  Valdez  Narrows,  including 
Bligh  Island  and  Glacier  Island  in  Prince 
William  Sound  and  extending  from  the 
Valdez  Arm  through  Valdez  Narrows  to 
points  including  the  land  masses  visible 
from  the  Port  of  Valdez.  (See  Figure 
1.)  The  visual  scene  included  all  navi¬ 
gational  aids  and  the  major  topogra¬ 
phical  features  necessary  for  pilotage 
through  Valdez  Narrows  to  the  Port  of 
Valdez. 


2.2.2  Ownship  Simulation 

The  handling  and  hydrodynamic  char¬ 
acteristics  of  a  165,000  DWT  tanker 
vessel  were  reproduced  on  the  CAORF 
simulator.  The  characteristics  of  the 
tanker  were  as  follows:  the  length  of 
the  tanker  was  951  feet  (290m),  and  the 
beam,  155  feet  (947m).  The  mean  draft 
for  the  ship  in  ballast  was  28  feet 
(8.5m).  These  plus  the  other  unique 
parameters  such  as  hydrodynamic  and 
aerodynamic  coefficients  were  utilized 
by  the  computer  program  to  produce 
realistic  and  accurate  motion  of  this 
particular  class  of  ship  (ownship). 


2.2.3  Bridge  Equipment 

The  CAORF  bridge,  with  equipment 
similar  to  that  normally  available  on  a 
tanker  of  165,000  DWT  includes: 


o  Radars 

o  Collision  avoidance  system 

o  Engine  throttle  -  bridge/engine 

room  control 

o  Gyro  repeater 

o  Communication  equipment 

o  Rudder  angle  indicators 

o  Speed  log 

o  Ship's  whistle 

o  Steering  stand 

o  Relative  wind  and  speed  indica¬ 

tors 

o  Rate  of  turn  indicator 

o  Two  gyro  repeaters  on  bridge 

wings 

o  Fathometer 

This  equipment  was  used  to  provide  ship 
control  and  navigation  capability.  Each 
responds  with  the  same  time  delays 
encountered  in  the  real  world. 


2.3  SCENARIO  DESIGN 

The  following  conditions  were  identical 
for  all  four  groups: 

Track  -  The  designated  track  is  shown 
in  Figure  2.  The  starting  point  A  to  B 
is  028°T  and  changes  to  045°T  at  point 
B  and  again  to  080  T  beyond  point  C. 

Wind  Conditions  -  Wind  intensity  varied 
from  40  knots,  randomly  gusting  to  50, 
at  extreme  point  A  to  30  knots  at 
Middle  Rock  and  beyond.  Wind  direc¬ 
tion  was  005°T  up  to  Middle  Rock  shift¬ 
ing  to  020°T  north  of  that  point. 
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Current  Conditions  -  The  current  speed 
was  held  constant  throughout  the  ex¬ 
perimental  runs  at  0.5  knots.  The  di¬ 
rection  of  the  current  was  that  of  the 
normal  outbound  Valdez  tide  and  had 
little  effect  on  the  ship. 

Visibility  and  Traffic  -  All  runs  in  the 
experiment  were  made  under  daylight 
conditions  with  unlimited  visibility.  No 
other  traffic  vessels  were  present  in 
the  harbor  during  any  runs. 


2.4  PROCEDURE 

Regardless  of  group,  each  subject  was 
briefed  about  the  nature  of  the  experi¬ 
ment.  As  previously  described,  they 
were  familiarized  with  both  the  bridge 
instrumentation  and  the  gaming  area. 
To  accomplish  this,  an  experienced 
CAORF  proctor  took  the  ship  through 
the  Valdez  Narrows  along  the  USCG 
track  line  as  shown  in  Figure  2.  Each 
group  of  four  successive  runs  was  made 
with  one  subject  from  each  of  four 
test-subject  groups.  A  subject  was 
randomly  chosen  and  allowed  to  transit 
the  track  as  indicated  (about  30 
minutes).  After  this  was  completed, 
the  subject  left  the  bridge  and  the 
second  subject  was  called  in.  This 
continued  until  all  four  subjects  had 
been  run  once.  The  first  subject  was 
then  called  for  his  second  run  and  so 
forth.  All  subjects  were  allowed  a  total 
of  five  trials. 


2.5  PERFORMANCE  MEASURES 

2.5.1  Average  Deviation  from  Re¬ 
ference  Track 

The  instantaneous  deviation  of  the 
ship's  c.g.  from  the  track  line  was 
recorded  each  24  seconds  during  a  run. 
Later,  the  average  deviation  for  the  run 
was  computed  from  these  data. 


2.5.2  Time  Out  of  Tolerance 

A  tolerance  of  +2.5  beam  widths  (+390 
feet)  was  superimposed  over  the  refer¬ 
ence  track  line.  As  the  ship's  actual 
track  line  is,*plotted,  measures  are  re¬ 
corded  as  to  both  the  frequency  (in 
terms  of  percent)  of  the  time  the  ship's 
position  was  within  tolerance  and  the 
frequency  of  the  time  the  ship's  posi¬ 
tion  was  out  of  the  tolerance.  These 
measures  indicated  how  well  the  mas¬ 
ters  maintained  vessel  position  with 
respect  to  the  reference  track  which 
they  were  required  to  follow. 


2.5.3  Rate  of  Acquisition 

In  addition  to  these  measures,  each 
subject's  performance  for  each  trial 
was  examined.  As  the  masters  pro¬ 
ceeded  through  the  five  runs,  their  per¬ 
formance  relative  to  time  out  of  toler¬ 
ance  and  deviation  from  track  with 
respect  to  the  reference  line  should  be 
expected  to  change,  illustrating  the 
benefits  of  practice  across  trials.  This 
effect  is  reflected  in  the  Analysis  of 
Variance  as  the  magnitude  of  the  F 
ratio  for  the  repeated  measure  (see 
Section  2.6,  Analysis  of  Data).  Subse¬ 
quent  contrasts  among  the  means  pro¬ 
vided  an  indication  of  the  differences  in 
the  rate  of  learning  between  each  of 
the  groups. 


2.6  Analysis  of  Data 

The  data  analyses  for  both  performance 
measures  were  based  on  a  two-between 
and  one-within  factors  mixed  design 
analysis  of  variance  (ANOVA)  for  re¬ 
peated  measures.  Table  2  shows  the 
data  matrix  for  such  a  design  as  applied 
to  the  present  investigation.  A  detailed 
ANOVA  is  given  in  Table  3  and  she.  s  a 
summary  of  the  comparisons  wh'ch 
were  used  to  test  for  main  effects  and 


TABLE  2.  DATA  MATRIX  FOR  TWO-BETWEEN  SUBJECTS 
ONE-WITHIN  SUBJECTS  ANALYSIS  OF  VARIANCE 


0 


interactions.  Any  significant  F  ratio  is 
accompanied  by  the  degree  of  confid¬ 
ence  that  the  difference  in  the  means 
tested  did  not  occur  by  chance.  That 


is,  a  p<  0.05  indicates  a  probability  of 
better  than  95%  that  the  differences 
being  tested  are  due  to  the  conditions 
of  the  experiment. 


TABLE  3.  ANOVA  -  TWO-BETWEEN  AND  ONE-WITHIN  SUBJECTS 

MIXED  DESIGN 


Source 

df 

MS 

F 

A 

(a-1)  =  1 

ss/i 

CL 

MVMSs(ab> 

B 

(b-1)  =  1 

SSb/l 

MSb/MSs(ab) 

AB 

(a-l)(b-l)  =  1 

SSab'‘ 

MSab/MSs(ab) 

S  (AB) 

abn  -  ab  =  16 

SW16 

— 

r 

(c— 1 )  =  4 

ssc/« 

MSc/MSs(ab)c 

AC 

(a-1)  (c-1)  =  4 

SSacrt 

MSac/MSs(ab)c 

BC 

(b-1)  (c-1)  =  4 

SSbc/4 

MSbc/MSs(ab)c 

ABC 

(a-1)  (b-1)  (c-1)  =  4 

SSabc^ 

MSabc/MSs(ab)c 

S(AB)C 

ab  (n-1)  (c-l)  =  64 

SSs(ab)c/64 

— 

TOTAL 

anbc  -  1  =  99 

A  =  Simulator  experiene,  2  levels 
B  =  Real-world  (Valdez)  exposure,  2  levels 
C  =  Trials,  5  levels 
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CHAPTER  3 


RESULTS 


3.1  OFF  TRACK  DEVIATION 

One  of  the  three  factors  yielded  signifi¬ 
cant  main  effects  while  thye  other  two 
tended  toward  significance1  as  shown  in 
Table  4.  Means  of  Off  Track  Deviation 
for  simulator  experience,  Valdez  exper¬ 
ience,  trials  and  the  four  experimental 
groups  over  trials  are  presented  in 
Appendix  B. 


Main  Effects 

(a)  Simulator  Experience  -  No  signifi¬ 
cant  difference  of  mean  off-track 


EXPERIENCE 

Figure  3.  Mean  Off  Track  Deviation 
for  Subjects  with  Simu¬ 
lator  and  No  Simulator 
Experience 


deviation  was  found  between  the  SIMU¬ 
LATOR  group  (x  =-56.79)  and  the  NO 
SIMULATOR  group  (x  =- 114.60)  when 
considered  independently  of  the  other 
two  factors.  While  not  statistically 
significant,  the  large  difference  in  the 
means  indicates  a  trend  toward  better 
trackkeeping  performance  in  the  SIMU¬ 
LATOR  group  (Figure  3). 

(b)  Valdez  Experience  -  The  VALDEZ 
group  produced  a  trackkeeping  per¬ 
formance  (x  =-50.76)  that  approaches 
significance  when  compared  with  the 
NO  VALDEZ  group  (x  =-120.63); 
F  (1,16)=3.0663,  p<.09;  Figure  4  shows 


Figure  4.  Mean  Off  Track  Deviation 
for  Subjects  with  Valdez 
and  No  Valdez  Experience 


The  commonly  accepted  significance  level  for  most  research  of  this  type  is  p<  .05. 
The  0.05  level  is  somewhat  arbitrary  since  any  level  of  significance  can  be 
interpreted  in  terms  of  a  probability  statement.  For  example,  a  significance  level 
of  p<  0.15  indicates  that  one  is  85%  confident  of  the  results  (as  compared  to  95% 
above)  or  that  there  is  a  15%  probability  that  the  results  occurred  by  chance  and  are 
not  due  to  the  conditions  of  the  experiment. 
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TABLE  4.  ANOVA  SOURCE  TABLE  FOR  OFF  TRACK  DEVIATION 


Source 

F 

Degrees  of 
Freedom 

Significance 

Simulator  vs. 

2.0988 

1,  16 

vO 

V 

a 

No  Sim  Experience 

Valdez  vs 

3.0663 

1,  16 

p<  .09 

No  Valdez  Experience 

Trials  l  -  5 

9.5526 

4,  64 

p<  .00004 

Simulator  x  Valdez 

2.1033 

1,  16 

p<  .15 

Simulator  x  Trials 

0.2734 

4,  64 

NS 

Valdez  x  Trials 

2.4607 

4,  64 

p<  .05 

Simulator  x  Valdez 

0.5468 

4,  64 

NS 

x  Trials 

these  data  considered  independently  of 
the  other  two  factors. 

(c)  Trials  -  A  significant  main  effect 
was  found  for  the  repeated  measure 
F(4,64)=:9.2 7,  p<.00l.  As  seen  in  Figure 
5,  subjects  improved  their  trackkeeping 
dramatically  in  the  first  two  trials  and 
continued  to  improve  more  gradually 
over  the  remaining  trials.  Thus,  across 
all  groups,  there  was  a  learning  effect 
present  in  the  repeated  trials. 

Interactions 

(a)  Simulator  Experience  and  Valdez 
Experience  -  The  interaction  between 
simulator  and  Valdez  was  not  statisti¬ 
cally  significant,  yet  a  trend  does  exist 
in  the  data  as  seen  in  Figure  6.  The 
major  contributer  to  this  effect  is  the 
NO  SIMULATOR/NO  VALDEZ  group. 
To  examine  this,  subsequent  compari¬ 
sons  were  performed. 


UJ 
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O  _J 
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<c  \— 
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f—  UJ 

60 

_j 
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U-  |— 

40 

O  UJ 
UJ 

u_ 

20 

0 

1  2  3  4!: 

TRIAL 

Figure  5.  Mean  Off  Track  Deviation 
for  Trials  1  through  5 


Figure  6.  Mean  Off  Track  Deviation 
for  Simulator  x  Valdez 
Interaction 


SIMULATOR/ 
NO  VALDEZ 
Gp  II 


NO  SIMULATOR/ 
NO  VALDEZ 
Gp  IV 


SIMULATOR/ 
VALDEZ 
Gp  I 


NO  SIMULATOR/ 
VALDEZ 
Gp  III 


Figure  7  presents  the  overall  means  of 
the  four  experimental  groups  for  the 
off-track  deviation  measure.  The  SIM¬ 
ULATOR/VALDEZ  (x  =-50.79)  group 
was  not  significantly  different  from 
either  the  SIMULATOR/NO  VALDEZ 
(x=-62.80)  or  the  NO  SIMULATOR/ 
VALDEZ  (x=-50.70)  group  using  the 
Duncan's  Multiple  Range  Test 
(Appendix  D,  Section  D.l).  It  was 
significantly  different  when  compared 
to  the  NO  SIMULATOR/NO  VALDEZ 
(x=  -178.47)  group,  p<.05,  which  is 
apparent  in  Figure  7. 

The  SIMULATOR/ NO  VALDEZ  group 
was  not  significantly  different  from  the 
NO  SIMULATOR/VALDEZ  group,  but 
was  statistically  different  from  the  NO 
SIMULATOR/NO  VALDEZ  group, 
p<.05,  using  the  Duncan's  Multiple 
Range  Test.  The  NO  SIMULATOR/ 
VALDEZ  group  was  also  significantly 
different  from  the  NO  SIMULATOR/NO 
VALDEZ  group,  p<.05.  Thus,  no  mean¬ 
ingful  difference  was  found  to  exist 
between  the  value  of  exposing  masters 
to  the  Narrows  on  the  simulator  or 


EXPERIENCE 

Figure  7.  Mean  Off  Track  Deviations 
for  the  Four  Experimental 
Groups 


in  the  real-world.  However  exposure  by 
either  means  was  beneficial. 

(b)  Simulator  Experience  and  Trials  - 
The  simulator  by  trials  interaction  was 
not  significant.  Thus,  the  learning 
effect  of  repeated  runs  was  not  shown 
to  be  more  prominent  for  the  NO  SIMU¬ 
LATOR  group  than  for  the  SIMULATOR 
group. 

(c)  Valdez  Experience  and  Trials  -The 
interaction  between  Valdez  experience 
and  trials  proved  significant;  F(4,64)  = 
2.4607,  p<  .05.  As  shown  in  Figure  8, 
those  groups  with  Valdez  exposure  im¬ 
prove  in  performance  substantially 
after  the  first  trial  and  then  reach 
asymptote  on  the  following  trials  indi¬ 
cating  that  some  limit  of  performance 
is  being  reached.  Those  groups  with  no 
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Figure  8.  Mean  Off  Track  Deviation 

for  Valdez  x  Trials  Interaction 


Valdez  exposure  initially  showed  a  de¬ 
crement  of  performance  in  trial  2  but 
then  continued  to  improve  in  perform¬ 
ance  in  trials  3,  4,  and  5  though  never 
reaching  the  same  level  of  performance 
as  those  groups  with  Valdez  exposure. 
It  is  evident  that  the  NO  VALDEZ 
groups  are  showing  continued  learning 
across  trials.  The  VALDEZ  groups  do 
not  show  such  an  effect.  It  is  these 
differences  which  resulted  in  a  signifi¬ 
cant  interaction. 

(d)  Simulator  Experience  and  Valdez 
Experience  by  Trials  -  This  interaction 
was  not  significant,  but  it  is  interesting 
to  examine  the  performance  of  the  four 
experimental  groups  (Figure  9).  Both 
SIMULATOR  groups  tend  to  approach 
the  asymptote  rather  rapidly  after  the 
third  trial,  whereas  the  NO 
SIMULATOR  groups  demonstrate  that 
learning  continues  through  trial  5. 
Comparisons  show  significant  differ¬ 
ences  exist  between  the  other  three 
groups  and  the  NO  SIMULATOR/NO 
VALDEZ  group  at  Trials  2  and  4 
(Appendix  D,  Section  D.  1.1  to  D.1.5). 


SIMULATOR/VALDEZ  Gp  I 
SIMULATOR/NO  VALDEZ  Gp  II 
NO  SIMULATOR/ VALDEZ  Gp  1 1 1 
NO  SIMULATOR/NO  VALDEZ  Gp  IV 


Figure  9.  Mean  Off  Track  Deviation 
for  Simulator  x  Valdez  x 
Trials  Interaction 

3.2  TIME  OUT  OF  TOLERANCE 

The  mean  percent  of  time  out  of  toler¬ 
ance  for  simulator  experience,  Valdez 
experience,  and  trials  for  the  four  ex¬ 
perimental  groups  were  calculated 
(Appendix  C).  Table  5  contains  the 
sources  of  variance  along  with  the 
significant  effects  for  these  measures. 
Details  of  these  results  are  discussed 
below: 


Main  Effects 

(a)  Simulator  Experience  -  A  non-signi¬ 
ficant  main  effect  exists  for  simulator 
experience  (x  =11.95)  compared  to  NO 
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Source 

F 

Degrees  of 
Freedom 

Significance 

Simulator  vs. 

2.1762 

1,  16 

p<  .11 

No  Sim  Experience 

Valdez  vs. 

3.6414 

1,  16 

p<  .07 

No  Valdez  Experience 

Trials  1  -  5 

8.5542 

4,  64 

p<.0001 

Simulator  x  Valdez 

3.5491 

1,  16 

p  <  .07 

Simulator  x  Trials 

0.0682 

4,  64 

NS 

Valdez  x  Trials 

3.6987 

4,  64 

p<  .009 

Simulator  x  Valdez 

0.8026 

4,  64 

NS 

x  Trials 

SIMULATOR  experience  (x  =19.76)  and 
is  illustrated  in  Figure  10. 

(b)  Valdez  Experience  -  The  main  effect 
for  Valdez  experience  approaches  signi¬ 
ficance.  Figure  1 1  shows  that  subjects 
with  Valdez  experience  (x  =10.80)  found 
themselves  out  of  tolerance  less  time 
than  those  subjects  with  no  Valdez  ex¬ 
perience  (x  =20.90);  F(l,16)  =  3.6414, 
p<  .07. 

(c) Trials  -  The  repeated  measure,  trials, 
is  once  again  highly  significant;  F(4,64) 
=  8.5542,  pc. 0001.  The  groups  all 
improved  in  their  efforts  to  stay  within 
tolerance  as  the  experiment  progressed 
(Figure  12). 

Interactions 
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(a)  Simulator  Experience  and  Valdez 
Experience  -  The  interaction  between 
simulator  and  Valdez  is  found  to  ap¬ 
proach  significance  and  is  presented  in 
Figure  14.  Comparisons  using  Duncan’s 


Figure  10.  Mean  Time  Out  of 

Tolerance  for  Subjects 
with  Simulator  and 
I Vo  Simulator  Experience 
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Figure  11.  Mean  Time  Out  of 

Tolerance  for  Subjects 
with  Valdez  and  No 
Valdez  Experience 


TRIAL 

Figure  12.  Mean  Time  Out  of 
Tolerance  for 
Trials  1  through  5 

(b)  Simulator  and  Trials  -  The  simulator 
by  trials  interaction  was  not  signifi¬ 
cant. 


Multiple  Range  Test  between  the  four 
experimental  groups  (Figure  13)  show 
no  differences  between  the  SIMULA- 
TOR/VALDEZ  (x  =11.88)  group  and 
either  the  SIMULATOR/NO  VALDEZ 
(x  =  12.02)  or  the  NO  SIMULATOR/ 
VALDEZ  (x  =9.72)  group.  (See  Ap¬ 
pendix  D,  Section  D.2.0).  However, 
the  comparison  with  the  NO  SIMULA¬ 
TOR/NO  VALDEZ  (x  =29.79)  group  was 
significant,  p<  .01.  The  SIMULATOR/ 
NO  VALDEZ  group  did  not  differ 
statistically  from  NO  SIMULATOR/ 
VALDEZ  group.  It  did  show  a 
significant  difference  when  compared 
to  the  NO  SIMULATOR/NO  VALDEZ 
group,  p<.01.  The  NO  SIMULATOR/ 
VALDEZ  group  was  also  statistically 
different  from  the  NO  SIMULATOR/NO 
VALDEZ  group,  p<.01.  Thus,  in 
considering  time  out  of  tolerance,  no 
appreciable  difference  could  be  as¬ 
certained  between  real-world  and 
simulator  exposure.  Either  exposure  or 
both  proved  almost  equally  valuable. 


(c)  Valdez  and  Trials  -  A  significant  in¬ 
teraction  was  found  between  Valdez 
experience  and  trials;  F(4,64)=3.6987, 
p<  .01.  As  can  be  seen  in  Figure  15, 
the  NO  VALDEZ  groups  degrade  in  per¬ 
formance  on  trial  2  and  then  begin  to 
show  improvement  while  the  VALDEZ 
groups  improve  quickly  and  continue 
until  trial  5,  where  they  degrade 
slightly.  Trials  3  to  5  for  this  group 
appear  to  represent  a  ceiling  effect  for 
keeping  the  ship  within  tolerance. 

(d)  Simulator  Experience  and  Valdez 
Experience  by  Trials  -  The  three-way 
interaction  was  not  significant.  The 
data  for  the  four  experimental  groups 
were  compared  in  Appendix  D  (Sections 
D.2.1  to  D.2.5)  and  are  represented  in 
Figure  16.  The  NO  SIMULATOR/NO 
VALDEZ  group  was  significantly  dif¬ 
ferent  from  the  other  three  groups  on 
all  trials.  It  is  interesting  to  note  that 
on  trial  2,  the  SIMULATOR/NO 
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Figure  15.  Mean  Time  Out  of 

Tolerance  for  Valdez 
x  Trials  Interactions 
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Figure  16.  Mean  Time  Out  of 

Tolerance  for  Simulator 
x  Valdez  x  Trials 
Interaction 
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CHAPTER  4 


DISCUSSION 


The  primary  objective  of  the  experi¬ 
ment  was  to  compare  the  performance 
of  masters  who  have  had  simulator  ex¬ 
perience  maneuvering  a  simulated 
VLCC  through  the  Valdez  Narrows  to 
the  Port  of  Valdez  to  that  of  masters 
who  have  had  teal-world  observational 
experience  maneuvering  VLCC's 
through  the  same  area.  In  addition, 
masters  who  have  had  neither  simulator 
nor  real-world  Valdez  experience  were 
compared  to  masters  who  have  had 
either  simulated  or  real-world  Valdez 
experience  or  both.  The  results  indi¬ 
cate  quite  clearly  that  performance  did 
not  differ  among  the  groups  having 
either  simulator  experience  only, 
Valdez  experience  only,  or  both.  How¬ 
ever,  each  of  these  three  groups  exhi¬ 
bited  significantly  better  performance 
than  those  subjects  who  had  neither 
simulated  nor  real-world  Valdez  experi¬ 
ence.  Therefore,  previous  simulation 
experience  and  previous  real-world  ex¬ 
posure  appear  to  influence  to  the  same 
degree  subsequent  performance  of 
masters  maneuvering  a  simulated  VLCC 
through  the  Valdez  Narrows.  In  a  like 
manner  we  can  verify  the  intuitive 
assumption  that  previous  experience 
transiting  the  Valdez  Narrows  should 
improve  performance  regardless  of 
simulator  exposure. 

Another  question  is  the  impact  of  simu¬ 
lators  as  a  valid  representation  of  real- 
world  experience.  For  example,  is  the 
training  received  on  a  simulator  equiva¬ 
lent  to  that  received  by  real-world  ex¬ 
posure  to  the  same  learning  objectives? 
A  comparison  between  the  performance 
scores  of  the  NO  SIMULATOR/VALDEZ 
and  SIMULATOR/NO  VALDEZ  groups 
would  reveal  if  any  such  differences 
exist.  If  the  performance  scores  of 


both  these  groups  were  identical  (i.e., 
statistically  not  significant),  it  may  be 
concluded  that  the  degree  of  ship¬ 
handling  performance  acquired  on  a 
simulator  is  equivalent  to  that  which 
can  be  acquired  from  observational 
experience  in  transiting  the  Narrows  in 
the  real  world.  The  comparison  of 
these  groups  is  shown  in  Figures  7  and 
13.  It  is  apparent  that  the  performance 
measures  for  both  the  NO  SIMULA¬ 
TOR/VALDEZ  group  and  the  SIMULA¬ 
TOR/NO  VALDEZ  group  are  identical. 
We  may  therefore  conclude  that 
masters  with  simulator  experience 
perform  as  well  as  those  who  have 
actually  made  frequent  transits  through 
the  Narrows. 

The  value  of  simulator  exposure  is  also 
shown  in  the  comparisons  of  perform¬ 
ance  data  between  those  masters  who 
have  had  experience  (i.e.,  subjects  who 
had  participated  in  the  operational  ex¬ 
ercises)  in  the  simulated  Valdez  gaming 
area  and  those  who  have  had  no  simula¬ 
tor  experience  in  the  gaming  area.  To 
evaluate  the  significance  of  simulator 
experience  alone,  the  appropriate  com¬ 
parisons  are  between  the  SIMULA¬ 
TOR/NO  VALDEZ  and  NO  SIMULA¬ 
TOR/NO  VALDEZ  groups.  These  data 
are  shown  in  Figures  7  and  13  and 
indicate  a  significant  difference  be¬ 
tween  the  mean  scores  of  both  groups. 
These  results  show  that  simulator  ex¬ 
perience  provides  a  major  contribution 
to  ship  control  measured  by  accuracy  of 
trackkeeping  in  the  Valdez  gaming 
area. 

The  value  of  simulator  experience  is 
even  more  strongly  shown  when  the 
SIMULATOR/VALDEZ  group  is  com¬ 
pared  to  the  SIMULATOR/NO  VALDEZ 
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group.  In  this  case,  we  are  able  to 
judge  the  combined  effect  of  both 
simulator  and  real-world  experience  to 
simulator  experience  alone.  Once  again 
it  is  evident  that  performance  measures 
for  these  groups  are  in  close  agreement 
with  each  other  (Figures  7  and  13),  and 
that  little  gain  in  trackkeeping  ability 
is  achieved  by  real-world  experience 
among  subjects  who  have  already  had 
exposure  to  simulator  operational  exer¬ 
cises. 

Another  finding  which  should  be  ad¬ 
dressed  is  the  learning  effect  across 
repeated  trials  on  the  Valdez  scenario. 
As  referenced  in  Tables  4  and  5,  both 
performance  measures  showed  highly 
significant  practice  effects  across 
trials.  The  specific  inter-trial  relation¬ 
ships  shown  in  Figures  5  and  12  indicate 
major  improvements  in  performance 
occurring  in  the  early  trials  and  a  grad¬ 
ually  leveling  off  with  additional  trials. 
The  most  significant  improvement 
occurs  in  trial  2.  However,  this  im¬ 
provement  does  not  occur  for  the  NO 
SIMULATOR/ NO  VALDEZ  group.  (See 
Figure  9.)  This  would  indicate  that  any 
learning  of  skills  on  a  simulator  is  rela¬ 
tively  stable  when  tested  at  a  later 
date.  Performance  peaked  after  the 
second  trial,  especially  for  the  SIMU¬ 
LATOR  /NO  VALDEZ  group  which 
represented  masters  who  had  partici¬ 
pated  in  the  Valdez  operational  exer¬ 
cises  but  did  not  have  any  similar  or 
real-world  experience  or  intervening 
simulator  exposure.  The  subjects  in 
this  group  originally  participated  in  the 
operational  exercises  anywhere  from 
six  months  to  two  years  prior  to  the 
present  study.  Figures  7  and  13,  show 
that  their  performance  is  the  same  as 
masters  who  had  intervening  Valdez  ex¬ 
perience. 

Apparently,  the  shiphandling  capabil¬ 
ities  acquired  through  simulation  alone 
have  been  retained  over  a  period  of 


time  without  intervening  practice  of 
the  specific  skills  either  on  a  simulator 
or  through  real-world  observation  expo¬ 
sure.  The  shiphandling  skills  developed 
in  the  operational  exercises  has  thus  far 
been  effective  in  making  this  six-month 
to  two-year  retention  possible.  It 
would  be  important,  however,  to  be 
able  to  assess  at  what  point  a  specific 
skill  begins  to  require  retraining  in 
order  to  maintain  a  particular  perform¬ 
ance  level. 

It  is  generally  assumed  that  refresher 
training  contributes  substantially  to  the 
long-term  retention  of  learned  skills.  A 
practical  application  would  be  the  case 
of  a  mariner  who  has  been  away  from 
duty  for  a  prolonged  period  of  time  or 
on  another  size  vessel  and  is  not  up  to 
his  previous  ability.  Such  an  individual 
could  profit  from  some  refresher  prac¬ 
tice  before  returning  to  duty.  A  simu¬ 
lator  with  sufficient  high  fidelity  could 
be  used  for  assessing  the  level  of  a 
mariner's  peformance  in  a  particular 
gaming  area,  and  could  provide  both  the 
practice  and  the  necessary  geographical 
familiarization  (e.g.  complexity  of  the 
harbor)  to  bring  a  person  up  to  previous 
levels  of  shiphandling  performance. 

Those  subjects  who  participated  in  the 
simulated  exercises  at  CAORF  can  be 
expected  to  show  a  decrement  in  per¬ 
formance  at  some  time  in  the  future  if 
they  have  no  opportunity  to  practice 
the  skills.  It  would  be  ideal  if  CAORF 
participants  who  had  no  real-world  ex¬ 
posure  to  the  Valdez  area  could  be 
brought  into  CAORF  periodically  for 
skill  assessment  and  evaluation  of  the 
amount  of  practice  necessary  to  restore 
an  ability  level.  This  would  provide 
valuable  information  about  when  and 
how  much  refresher  practice  would  be 
needed  for  a  particular  harbor.  The 
likelihood  of  such  an  evaluation  is  not 
good  since  most  of  the  masters  who 
participated  in  the  exercises  hope  that 
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they  will  be  able  to  apply  their  skills  in 
Valdez.  It  was  difficult  enough  to 
obtain  the  captains'  participation  in  the 
present  experiment  since  so  many 
mariners  who  have  been  through  the 
operational  exercises  have  subsequently 
been  exposed  to  Valdez. 

It  is  in  order  here  to  suggest  future 
experimentation  that  addresses  the  pro¬ 
blem  of  retention  and  retraining. 
Several  groups  of  masters  could  be  re¬ 
cruited  to  participate  in  a  structured 
training  program.  Periodic  simulator 
evaluation  of  the  amount  of  retraining 
needed  to  re-establish  the  ability  level 
attained  during  the  original  training 
could  be  planned  for  several  different 
intervals  (e.g.  every  six  months  versus 
every  year).  The  different  intervals 


could  be  examined  for  long-term  reten¬ 
tion  effectiveness,  which  could  provide 
information  that  would  establish  cri¬ 
tical  time  periods  for  skill  assessment 
and  subsequent  retraining. 


In  summary,  the  present  investigation 
has  demonstrated  the  validity  of  simu¬ 
lator  experience  as  used  in  previous 
exercises  and  research.  It  has  been 
shown  to  be  an  effective  tool  in  im¬ 
proving  performance  regardless  of  a 
subject's  previous  exposure  to  real- 
world  experiences.  The  results  of  this 
investigation  serve  to  validate  the 
Valdez  Narrows  operational  exercises 
as  formulated  at  CAORF  and  in  general 
represents  a  cost-efficient  method  of 
improving  shiphandling  skills. 
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APPENDIX  A 


THE  COMPUTER  AIDED  OPERATIONS  RESEARCH  FACILITY  (CAORF) 


A.I  DESCRIPTION  OF  CAORF 

CAORF  is  the  sophisticated  ship- 
maneuvering  simulator  operated  by  the 
U.S.  Maritime  Administration  for  con¬ 
trolled  research  into  man-ship-environ¬ 
ment  problems.  Controlled  experi¬ 
ments,  which  might  require  several  ves¬ 
sels,  cannot  be  performed  readily  in  the 
real  world  and  would  certainly  be  ruled 
out  for  testing  situations  that  involve 
potential  danger.  Such  experiments  can 
be  performed  safely  and  easily  at 
CAORF.  A  simplified  cutaway  of  the 
simulator  building  is  shown  in  Figure 
A-l  and  the  relationships  among  the 
major  subsystems  are  illustrated  in 
Figure  A-2. 

All  actions  called  for  by  the  watch 
officer  on  the  bridge  are  fed  through  a 
central  computer  that  alters  the  visual 
scene  and  all  bridge  displays  and  re¬ 
peaters  in  accordance  with  the  calcu¬ 
lated  dynamic  response  of  ownship  and 
the  environmental  situation  being 
simulated.  CAORF  has  the  capability 
of  simulating  any  ship,  port,  or  area  in 
the  world.  The  major  subsystems  are: 


o  Wheelhouse,  which  contains  all 
equipment  and  controls  needed  by 
the  test  subject  watch  officer  to 
maneuver  ownship  through  a  sce¬ 
nario,  including  propulsion  and 
steering  controls,  navigational 
equipment  and  communication 
gear. 

o  Central  Data  Processor,  which 
computes  the  motion  of  ownship 
in  accordance  with  its  known 
characteristics,  models  the  be¬ 
havior  of  all  other  traffic  ships, 


and  drives  the  appropriate  bridge 
indicators. 

o  Image  Generator,  which  con¬ 

structs  the  computer-generated 
visual  image  of  the  surrounding 
environment  and  traffic  ships  that 
is  projected  onto  a  cylindrical 
screen  for  visual  realism. 

o  Radar  Signal  Generator,  which 

synthesizes  video  signals  to  stim¬ 
ulate  the  bridge  radars  and  colli¬ 
sion  avoidance  system  for  the  dis¬ 
play  of  traffic  ships  and  surround¬ 
ing  environment. 

o  Control  Station,  from  which  the 

experiment  is  started  and  stop¬ 
ped,  traffic  ships  and  environment 
can  be  controlled,  mechanical 
failures  can  be  introduced,  and 
external  communications  with 
ownship's  bridge  can  be  simulated. 

o  Human  Factors  Monitoring  Station, 

from  which  unobtrusive  observa¬ 
tion  and  video  recording  of  test 
subject  behavior  can  be  carried 
out  by  experimental  psycholog¬ 
ists. 


A.2  SIMULATED  BRIDGE 

The  simulated  bridge  consists  of  a 
wheelhouse  20  feet  (6.1  m)  wide  and  14 
feet  (4.3  m)  deep.  The  equipment  on 
the  CAORF  bridge  is  similar  to  that 
normally  available  in  the  merchant 
fleet  and  responds  with  realistically 
duplicated  time  delays  and  accuracy. 
The  arrangement  is  based  on  contem¬ 
porary  bridge  design  and  includes  the 
following  equipment: 


A-l 


HUMAN  FACTORS 
MONITORING  STATION 


Figure  A-l.  Cutaway  of  CAORF  Building 


Steering  Controls  And  Displays  -  a 

gyropilot  helm  unit  with  standard 
steering  modes,  rate  of  turn  indi¬ 
cator,  rudder  angle/rudder  order 
indicators,  and  gyro  repeaters. 

Propulsion  Controls  and  Displays  - 

an  engine  control  panel  (capable 
of  simulating  control  from  either 
bridge  or  engine  room)  containing 
a  combined  engine  order  tele¬ 
graph/throttle,  an  rpm  indicator 
and  a  switch  for  selecting  the 
operating  mode,  such  as  finished 
with  engine,  warm  up,  maneuver¬ 
ing  and  sea  speed. 

Thruster  Controls  and  Displays  - 

bow  and  stern  thrusters  and  their 


respective  indicators  and  status 
lights. 

o  Navigation  Systems  -  two  radars 

capable  of  both  relative  and  true 
motion  presentations,  plus  a  colli¬ 
sion  avoidance  system.  Capabil¬ 
ity  exists  for  future  additions 
such  as  a  digital  fathometer, 
Radio  Direction  Finder,  and  Loran 
C  and  Omega  systems. 

o  Communications  -  simulated 

VHF/SSB  radio,  docking  loud¬ 
speaker  (talkback)  system,  sound 
powered  phones  and  ship's  whistle. 

o  Wind  Indicators  -  indicate  to  the 

bridge  crew  the  true  speed  and 
direction  of  simulated  wind. 
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Figure  A-2.  Major  CAORF  Subsystems 


A.3  OWNSH1P  SIMULATION 


A.4  IMAGE  GENERATION 


Any  ship  can  be  simulated  at  CAORF.  The  visual  scene  is  generated  at 

The  computerized  equations  of  motion  CAORF  to  a  degree  of  realism  suffi- 

are  adapted  to  the  ship  by  changing  cient  for  valid  simulation.  The  scene 

specific  coefficients,  among  which  are  (Figure  A-3)  includes  all  the  man-made 

hydrodynamic,  inertial,  propulsion,  structures  and  natural  components  of 

thruster,  rudder,  aerodynamic,  etc.  the  surrounding  scene  that  mariners 

Wind  and  currents  realistically  affect  familiar  with  the  geographical  area 

ship  motion  according  to  draft  (loaded  deem  necessary  as  cues  for  navigation, 

or  ballasted)  and  relative  speed  and 

direction.  Ownship's  computer  model  Thus,  bridges,  buoys,  lighthouses,  tall 

was  validated  by  comparing  various  buildings,  mountains,  glaciers,  piers, 

simulated  maneuvers  (e.g.,  zig-zag,  coastlines,  and  islands  would  be  de¬ 
turning  circle,  spiral,  crash  stop,  and  picted  in  the  scene.  In  addition,  the 

acceleration  tests)  with  sea  trail  data.  closest  traffic  ships  and  the  forebody  of 


Figure  A-3.  Typical  Simulated  Visual  Scene  at  CAORF 


ownship  appear.  All  elements  in  the 
scene  appear  to  move  in  response  to 
ownship's  maneuvers.  The  sky  is  de¬ 
picted  without  clouds  and  the  water 
without  waves. 

For  enhanced  realism  the  scene  is  pro¬ 
jected  in  full  color.  The  perspective  is 
set  for  the  actual  bridge  height  above 
waterline  for  the  simulated  ship.  Sha¬ 
dowing  can  be  varied  according  to  the 
position  of  the  sun  at  different  times  of 
day. 

Environmental  conditions  also  affect 
the  scene.  The  lighting  can  be  varied 
continuously  from  full  sun  to  moonless 
night.  At  night,  lights  can  be  seen  on 
traffic  vessels,  buoys,  piers,  and  other 
points  ashore.  Visibility  in  the  day  or 
night  can  be  reduced  to  simulate  any 
degree  of  fog  or  haze. 


A.5  RADAR  SIGNAL  GENERATION 

The  Radar  Signal  Generator  produces 
real-time  video  signals  for  driving  the 
two  radar  PPIs.  The  items  displayed 
are  synchronized  with  the  visual  scene 
and  include  navigation  aids,  ships, 
shorelines  and  other  topographical 
features  with  appropriate  target 
shadowing,  clutter,  range  attenuation, 
and  receiver  noise.  The  radar  gaming 
area,  which  covers  an  area  of  150  by 
200  miles,  extends  beyond  the  visual 
gaming  area,  which  is  50  by  100  miles. 
Within  the  radar  gaming  area,  as  many 


as  40  moving  traffic  ships  can  be 
displayed.  The  radar  signal  generator 
also  drives  the  collision  avoidance 
system,  which  can  be  slaved  to  either 
of  the  master  PPIs. 


A.6  CONTROL  STATION 

The  Control  Station  (Figure  A-4)  is  the 
central  location  from  which  the  simula¬ 
tor  experiment  is  controlled  and  moni¬ 
tored.  An  experiment  can  be  initiated 
anywhere  within  the  visual  gaming  area 
with  any  ship  traffic  configuration. 
The  Control  Station  enables  the  re¬ 
searchers  to  interface  with  the  watch¬ 
standing  crew  on  the  bridge,  to  simu¬ 
late  malfunctions,  and  to  control  the 
operating  mode  of  the  simulator.  The 
Control  Station  is  also  capable  of  con¬ 
trolling  motions  of  traffic  ships  and 
tugs  in  the  gaming  area  and  simulating 
telephone,  intercom,  radio  (VHF,  SSB) 
and  whistle  contact  with  the  CAORF 
bridge  crew. 


A.7  HUMAN  FACTORS  MONITOR¬ 
ING  STATION 

The  Human  Factors  Monitoring  Station 
(Figure  A-5)  is  designed  to  allow  collec¬ 
tion  of  data  on  crew  behavior.  Moni¬ 
toring  data  is  provided  by  five  closed- 
circuit  TV  cameras  and  four  micro¬ 
phones  strategically  located  throughout 
the  wheelhouse  to  record  all  activities, 
comments  and  commands. 


A-5 
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Figure  A- 5.  Human  Factors  Monitoring  Station 
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APPENDIX  B 


OFF  TRACK  DEVIATION 


TABLE  B-i.  OFF  TRACK  DEVIATION  MEANS  FOR 
SIMULATOR  EXPERIENCE,  VALDEZ  EXPERIENCE,  AND  TRIALS* 


Trials 

Group 

All 

1 

2 

3 

4 

5 

All  Subjects 

- 

-177.7 

-107.3 

-72.3 

-40.2 

-30.8 

Simulator 

-56.7 

-165.5 

-70.3 

-44.7 

-4.5 

1.1 

No  Simulator 

-114.6 

-189.7 

-144.3 

-100.0 

-75.9 

-62.8 

Valdez 

-50.7 

-176.5 

-24.1 

-40.6 

-3.6 

-8.8 

No  Valdez 

-120.6 

-178.7 

-109.5 

-104.1 

-76.8 

-52.0 

Simulator/ 

Valdez 

-50.7 

-210.5 

-29.8 

-33.0 

15.4 

9.1 

Simulator/ 

No  Valdez 

-62.7 

-120.5 

-110.7 

-56.3 

-24.4 

-1.7 

No  Simulator/ 
Valdez 

-50.7 

-142.5 

-18.4 

-48.1 

-22.7 

-21.7 

No  Simulator/ 

No  Valdez 

-178.4 

-236.9 

-270.2 

-151.9 

-129.1 

-103.9 

*  Negative  Numbers  =  Feet  left  of  track; 
Positive  =  Feet  right  of  track 


APPENDIX  C 


PERCENT  OF  TIME  OUT  OF  TOLERANCE 


TABLE  C-l.  MEAN  PERCENT  OF  TIME  OUT  OF  TOLERANCE  FOR 
SIMULATOR  EXPERIENCE,  VALDEZ  EXPERIENCE  AND  TRIALS 


Trial 

Group 

All 

1 

2 

3 

4 

5 

All  Subjects 

- 

11.9 

19.4 

15.2 

11.1 

9.6 

Simulator 

11.9 

19.4 

16.0 

12.0 

6.9 

5.2 

No  Simulator 

19.7 

27.4 

23.2 

18.5 

15.4 

14.1 

Valdez 

10.8 

22.4 

9.8 

8.2 

5.3 

7.6 

No  Valdez 

20.9 

23.9 

29.5 

22.3 

17.0 

11.7 

Simulator/ 

Valdez 

11.8 

23.8 

13.5 

7.4 

6.6 

7.0 

Simulator/ 

No  Valdez 

12.0 

14.9 

18.6 

16.6 

7.2 

1.5 

No  Simulator/ 
Valdez 

9.7 

21.9 

6.1 

9.0 

4.0 

7.3 

No  Simulator/ 

No  Valdez 

29.7 

32.9 

40.4 

27.9 

26.8 

20.9 

APPENDIX  D 


MULTIPLE  RANGE  STATISTICS 


D.l  DUNCAN'S  MULTIPLE  RANGE  STATISTIC  FOR  OFF  TRACK  DEVIATION 
D.1.0  Overall  Group  Means  + 


Simulator/ 

Valdez 

No  Simulator/ 
Valdez 

Simulator/ 
No  Valdez 

No  Simulator/ 
No  Valdez 

X1 

X3 

\ 

Means 

-50.794 

-50.733 

-62.797 

-178.469 

X1 

- 

.061 

12.003 

127.675* 

X2 

- 

12.064 

127.736* 

X3 

- 

115.672* 

X4 

- 

*p<  .05 


t  Differences  between  each  pair  of  means  are  presented  here. 
Significant  differences  are  noted  with  asterisk. 

D.1.1  Group  Means  at  Trial  1 


Simulator/  No  Simulator/  Simulator/  No  Simulator/ 
No  Vaidez  Valdez  Valdez  No  Valdez 


*2 


Means 


-120.571 


-142.534 


-210.542 


-236.983 


0.1.2  Group  Means  at  Trial  2 


No  Simulator/ 
Valdez 

X3 

Simulator/ 

Valdez 

X1 

Simulator/ 

No  Valdez 

Jo  Simulator/ 
No  Valdez 

Means 

-18.445 

-29.864 

-110.768 

-270.836* 

X3 

- 

11.419 

92.323 

251.836* 

X1 

X1 

X4 

80.904 

240.417* 

159.513* 

*p<  .01 


D.1.3  Group  Means  at  Trial  3 


Simulator/ 

Valdez 

X1 

No  Simulator/ 
Valdez 

X3 

Simulator/ 
No  Valdez 

No  Simulator/ 
No  Valdez 

Means 

-33.0381 

-48.175 

-56.380 

-151.9866 

X1 

- 

15.1369 

23.342 

118.9485 

X3 

X2 

X4 

8.205 

103.8116 

95.6066 

NS 
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D.1.4  Group  Means  at  Trial  4 

Simulator/ 

No  Simulator/ 

Simulator/ 

No  Simulator/ 

Valdez 

Valdez 

No  Valdez 

No  Valdez 

Means 

15.440 

-22.754 

-24.490 

-129.162 

X1 

Ij 

38.194 

39.93 

144.602* 

X3 

X2 

X4 

D.1.5  Group  Means  at  Trial  5 

1.736 

106.408 

104.672 

*p<  .01 

Simulator/ 

Simulator/ 

No  Simulator/ 

No  Simulator/ 

Valdez 

No  Valdez 

Valdez 

No  Valdez 

21.756 

-103.937 

25.789 

107.97 

19.979 

102.16 

82.181 

D.2  DUNCAN'S  MULTIPLE  RANGE  STATISTIC  FOR  TIME  OUT  OF  TOLERANCE 


D.2.0  Overall  Group  Means 


No  Simulator/ 
Valdez 


Simulator/ 

Valdez 


Simulator/ 
No  Valdez 


No  Simulator/ 
No  Valdez 


9.724 

11.887 

12.016 

29.795 

- 

2.163 

2.292 

20.071* 

- 

.129 

17.908* 

_ 

17.779* 

No  Simulator 
No  Valdez 


D.2.2  Group  Means  at  Trial  2 


No  Simulator/  Simulator/  Simulator/  No  Simulator/ 
Valdez  Valdez  No  Valdez  No  Valdez 

X3  Xj  X2  X4 


Means 

6.120 

13.560 

18.625 

40.422 

X3 

- 

7.44 

12.505* 

34.302** 

X1 

- 

5.065 

26.862** 

X2 

- 

21.797** 

X4 

*p<  .05 

**p<  .01 

D.2.3  Group  Means  at  Trial  3 


Simulator/ 

Valdez 

X1 

No  Simulator/ 
Valdez 

X3 

Simulator/ 
No  Valdez 

No  Simulator/ 
No  Valdez 

Means 

7.478 

9.09 

16.664 

27.939 

X1 

- 

1.612 

9.186 

20.461* 

X3 

X2 

X4  | 

7.574 

18.849* 

11.275 

*p<.01 

D-5 


D.2.4  Group  Means  at  Trial  4 


No  Simulator/ 
Valdez 

X3 

Simulator/ 

Valdez 

X1 

Simulator/ 
No  Valdez 

*2 

No  Simulator/ 
No  Valdez 

X4 

Means 

4.05 

6.622 

7.238 

26.853 

X3 

- 

2.572 

3.188 

22.803* 

X1 

X2 

.616 

20.231* 

19.615* 

*p<  .01* 


D.2.5  Group  Means  at  Trial  5 


Simulator/ 

No  Simulator/ 

Simulator/ 

No  Simulator/ 

No  Valdez 

Valdez 

Valdez 

No  Valdez 

*2 

S 

xi 

Means 

2.596 

7.38 

7.924 

20.8360 

